Zone-based federated learning

ABSTRACT

A method for managing model updates by a first network device includes receiving, at the first network device associated with a first zone model of multiple zone models, a global model from a second network device associated with the global model. The method also includes transmitting, from the first network device, the global model to user equipment (UEs) in a first group of UEs associated with the first zone model, a different group of UEs associated with each of the plurality of zone models. The method further includes receiving, at the first network device, weights associated with the global model from each UE in the first group. The method still further includes updating, at the first network device, the first zone model based on the received weights. The method also includes transmitting, from the first network device, the updated first zone model to each UE in the first group.

FIELD OF THE DISCLOSURE

Aspects of the present disclosure generally relate to wirelesscommunications, and more particularly to techniques and apparatuses forzone-based federated learning.

BACKGROUND

Wireless communication systems are widely deployed to provide varioustelecommunication services such as telephony, video, data, messaging,and broadcasts. Typical wireless communication systems may employmultiple-access technologies capable of supporting communication withmultiple users by sharing available system resources (e.g., bandwidth,transmit power, and/or the like). Examples of such multiple-accesstechnologies include code division multiple access (CDMA) systems, timedivision multiple access (TDMA) systems, frequency-division multipleaccess (FDMA) systems, orthogonal frequency-division multiple access(OFDMA) systems, single-carrier frequency-division multiple access(SC-FDMA) systems, time division synchronous code division multipleaccess (TD-SCDMA) systems, and long term evolution (LTE).LTE/LTE-Advanced is a set of enhancements to the universal mobiletelecommunication system (UMTS) mobile standard promulgated by the ThirdGeneration Partnership Project (3GPP).

A wireless communication network may include a number of base stations(BSs) that can support communication for a number of user equipment(UEs). A user equipment (UE) may communicate with a base station (BS)via the downlink and uplink. The downlink (or forward link) refers tothe communication link from the BS to the UE, and the uplink (or reverselink) refers to the communication link from the UE to the BS. As will bedescribed in more detail, a BS may be referred to as a Node B, a gNB, anaccess point (AP), a radio head, a transmit receive point (TRP), a newradio (NR) BS, a 5G Node B, and/or the like.

The above multiple access technologies have been adopted in varioustelecommunication standards to provide a common protocol that enablesdifferent UEs to communicate on a municipal, national, regional, andeven global level. New Radio (NR), which may also be referred to as 5G,is a set of enhancements to the LTE mobile standard promulgated by theThird Generation Partnership Project (3GPP). NR is designed to bettersupport mobile broadband Internet access by improving spectralefficiency, lowering costs, improving services, making use of newspectrum, and better integrating with other open standards usingorthogonal frequency division multiplexing (OFDM) with a cyclic prefix(CP) (CP-OFDM) on the downlink (DL), using CP-OFDM and/or SC-FDM (e.g.,also known as discrete Fourier transform spread OFDM (DFT-s-OFDM)) onthe uplink (UL), as well as supporting beamforming, multiple-inputmultiple-output (MIMO) antenna technology, and carrier aggregation.

Artificial neural networks may comprise interconnected groups ofartificial neurons (e.g., neuron models). The artificial neural networkmay be a computational device or represented as a method to be performedby a computational device. A deep neural network may be an example of anartificial neural network. Deep neural networks may improve privacy orimplement Internet of Things (IoT) services. In some examples, a deepneural network may be trained in a decentralized manner. As an example,the deep neural network may be trained in multiple edge devices, such asa mobile device, where local data samples may be specified as trainingdata. It may be desirable to customize deep neural networks based oninherent similarities and differences found among the edge devicesspecified for training.

SUMMARY

In one aspect of the present disclosure, a method for managing modelupdates by a first network device includes receiving, at the firstnetwork device associated with a first zone model of a number of zonemodels, a global model from a second network device associated with theglobal model. The method further includes transmitting, from the firstnetwork device, the global model to UEs in a first group of UEsassociated with a first zone model. In some examples, a different groupof UEs may associated with each of the number of zone models. The methodstill further includes receiving, at the first network device, weightsassociated with the global model from each UE in the first group. Themethod also includes updating, at the first network device, the firstzone model based on the received weights. The method further includestransmitting, from the first network device, the updated first zonemodel to each UE in the first group.

Another aspect of the present disclosure is directed to an apparatus formanaging model updates at a first network device. The apparatus includesmeans for receiving, at the first network device associated with a firstzone model of a number of zone models, a global model from a secondnetwork device associated with the global model. The apparatus furtherincludes means for transmitting, from the first network device, theglobal model to UEs in a first group of UEs associated with the firstzone model. In some examples, a different group of UEs may associatedwith each of the number of zone models. The apparatus still furtherincludes means for receiving, at the first network device, weightsassociated with the global model from each UE in the first group. Theapparatus also includes means for updating, at the first network device,the first zone model based on the received weights. The apparatusfurther includes means for transmitting, from the first network device,the updated first zone model to each UE in the first group.

In another aspect of the present disclosure, a non-transitorycomputer-readable medium with non-transitory program code recordedthereon for managing model updates at a first network device isdisclosed. The program code is executed by a processor and includesprogram code to receive, at the first network device associated with afirst zone model of a number of zone models, a global model from asecond network device associated with the global model. The program codefurther includes program code to transmit, from the first networkdevice, the global model to UEs in a first group of UEs associated witha first zone model. In some examples, a different group of UEs mayassociated with each of the number of zone models. The program codestill further includes program code to receive, at the first networkdevice, weights associated with the global model from each UE in thefirst group. The program code also includes program code to update, atthe first network device, the first zone model based on the receivedweights. The program code further includes program code to transmit,from the first network device, the updated first zone model to each UEin the first group.

Another aspect of the present disclosure is directed to an apparatus formanaging model updating at a first network device. The apparatusincludes a processor; a memory coupled with the processor; andinstructions stored in the memory and operable, when executed by theprocessor, to cause the apparatus to receive, at the first networkdevice associated with a first zone model of a number of zone models, aglobal model from a second network device associated with the globalmodel. Execution of the instructions further cause the apparatus totransmit, from the first network device, the global model to UEs in afirst group of UEs associated with a first zone model. In some examples,a different group of UEs may associated with each of the number of zonemodels. Execution of the instructions also cause the apparatus toreceive, at the first network device, weights associated with the globalmodel from each UE in the first group. Execution of the instructionsstill further cause the apparatus to update at the first network device,the first zone model based on the received weights. Execution of theinstructions further cause the apparatus to configured to transmit, fromthe first network device, the updated first zone model to each UE in thefirst group.

In one aspect of the present disclosure, a method for training modelsperformed at a UE includes receiving, at the first UE associated with afirst group of UEs, a first model from a first network device associatedwith a first zone model of a number of zone models. In some examples,the first group of UEs is associated with a first zone model, and adifferent group of UEs are associated with each of the number of zonemodels. The method further includes identifying, at the first UE, anetwork device for training the first model based on one or both of acurrent connectivity state of the UE or a current resource use of theUE. The method still further includes transmitting, to the first networkdevice, model weight updates based on the training of the first model.The method also includes receiving, from the first network device, thefirst zone model based on the transmitted model weights updates.

Another aspect of the present disclosure is directed to an apparatus fortraining models at a UE. The apparatus includes means for receiving, atthe first UE associated with a first group of UEs, a first model from afirst network device associated with a first zone model of a number ofzone models. In some examples, the first group of UEs is associated witha first zone model, and a different group of UEs are associated witheach of the number of zone models. The apparatus further includes meansfor identifying, at the first UE, a network device for training thefirst model based on one or both of a current connectivity state of theUE or a current resource use of the UE. The apparatus still furtherincludes means for transmitting, to the first network device, modelweight updates based on the training of the first model. The apparatusalso includes means for receiving, from the first network device, thefirst zone model based on the transmitted model weights updates.

In another aspect of the present disclosure, a non-transitorycomputer-readable medium with non-transitory program code recordedthereon for training models at a UE is disclosed. The program code isexecuted by a processor and includes program code to receive, at thefirst UE associated with a first group of UEs, a first model from afirst network device associated with a first zone model of a number ofzone models. In some examples, the first group of UEs is associated witha first zone model, and a different group of UEs are associated witheach of the number of zone models. The program code further includesprogram code to identify, at the first UE, a network device for trainingthe first model based on one or both of a current connectivity state ofthe UE or a current resource use of the UE. The program code stillfurther includes program code to transmit, to the first network device,model weight updates based on the training of the first model. Theprogram code also includes program code to receive, from the firstnetwork device, the first zone model based on the transmitted modelweights updates.

Another aspect of the present disclosure is directed to an apparatus fortraining models at a UE. The apparatus includes a processor; a memorycoupled with the processor; and instructions stored in the memory andoperable, when executed by the processor, to cause the apparatus toreceive, at the first UE associated with a first group of UEs, a firstmodel from a first network device associated with a first zone model ofa number of zone models. In some examples, the first group of UEs isassociated with a first zone model, and a different group of UEs areassociated with each of the number of zone models. Execution of theinstructions further cause the apparatus to identify, at the first UE, anetwork device for training the first model based on one or both of acurrent connectivity state of the UE or a current resource use of theUE. Execution of the instructions still further cause the apparatus totransmit, to the first network device, model weight updates based on thetraining of the first model. Execution of the instructions also causethe apparatus to receive, from the first network device, the first zonemodel based on the transmitted model weights updates.

According to another aspect of the present disclosure, a non-transitorycomputer readable medium storing program code for wirelesscommunications by a BS includes program code to determine a timingcondition for a UE. The base station also includes program code totransmit a number of HP grants for scheduling a number of HP uplinktransmissions in a slot. The base station also includes program code totransmit a LP grant for scheduling an LP uplink transmission in theslot. The LP uplink transmission overlaps one or more of the number ofHP uplink transmissions in the slot. A time between one of the number ofHP grants and one of the HP uplink transmissions overlapping thescheduled LP uplink transmission satisfying the timing condition. Thebase station further includes program code to receive a multiplexedcommunication during one of the scheduled HP uplink transmissions.

The foregoing has outlined rather broadly the features and technicaladvantages of examples according to the disclosure in order that thedetailed description that follows may be better understood. Additionalfeatures and advantages will be described. The conception and specificexamples disclosed may be readily utilized as a basis for modifying ordesigning other structures for carrying out the same purposes of thepresent disclosure. Such equivalent constructions do not depart from thescope of the appended claims. Characteristics of the concepts disclosed,both their organization and method of operation, together withassociated advantages will be better understood from the followingdescription when considered in connection with the accompanying figures.Each of the figures is provided for the purposes of illustration anddescription, and not as a definition of the limits of the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

So that features of the present disclosure can be understood in detail,a particular description may be had by reference to aspects, some ofwhich are illustrated in the appended drawings. It is to be noted,however, that the appended drawings illustrate only certain aspects ofthis disclosure and are therefore not to be considered limiting of itsscope, for the description may admit to other equally effective aspects.The same reference numbers in different drawings may identify the sameor similar elements.

FIG. 1 is a block diagram conceptually illustrating an example of awireless communication network, in accordance with various aspects ofthe present disclosure.

FIG. 2 is a block diagram conceptually illustrating an example of a basestation in communication with a user equipment (UE) in a wirelesscommunication network, in accordance with various aspects of the presentdisclosure.

FIG. 3 illustrates an example implementation of designing a neuralnetwork using a system-on-a-chip (SOC), including a general-purposeprocessor, in accordance with certain aspects of the present disclosure.

FIGS. 4A, 4B, and 4C are diagrams illustrating a neural network, inaccordance with aspects of the present disclosure.

FIG. 4D is a diagram illustrating an exemplary deep convolutionalnetwork (DCN), in accordance with aspects of the present disclosure.

FIG. 5 is a block diagram illustrating an exemplary deep convolutionalnetwork (DCN), in accordance with aspects of the present disclosure.

FIG. 6 is a diagram illustrating an example of different zones in afederated learning system, in accordance with aspects of the presentdisclosure.

FIG. 7 is a timing diagram illustrating an example of customizing zonemodels, in accordance with aspects of the present disclosure.

FIG. 8 is a block diagram illustrating an example of a participatingdevice including a federated learning (FL) manager, in accordance withaspects of the present disclosure.

FIG. 9 is a timing diagram illustrating an example of federated learningwith a training proxy, in accordance with aspects of the presentdisclosure.

FIG. 10 is a flow diagram illustrating an example process performed, forexample, by a network device, in accordance with various aspects of thepresent disclosure.

FIG. 11 is a flow diagram illustrating an example process performed, forexample, by a UE, in accordance with various aspects of the presentdisclosure.

DETAILED DESCRIPTION

Various aspects of the disclosure are described more fully below withreference to the accompanying drawings. This disclosure may, however, beembodied in many different forms and should not be construed as limitedto any specific structure or function presented throughout thisdisclosure. Rather, these aspects are provided so that this disclosurewill be thorough and complete, and will fully convey the scope of thedisclosure to those skilled in the art. Based on the teachings oneskilled in the art should appreciate that the scope of the disclosure isintended to cover any aspect of the disclosure, whether implementedindependently of or combined with any other aspect of the disclosure.For example, an apparatus may be implemented or a method may bepracticed using any number of the aspects set forth. In addition, thescope of the disclosure is intended to cover such an apparatus ormethod, which is practiced using other structure, functionality, orstructure and functionality in addition to or other than the variousaspects of the disclosure set forth. It should be understood that anyaspect of the disclosure disclosed may be embodied by one or moreelements of a claim.

Several aspects of telecommunication systems will now be presented withreference to various apparatuses and techniques. These apparatuses andtechniques will be described in the following detailed description andillustrated in the accompanying drawings by various blocks, modules,components, circuits, steps, processes, algorithms, and/or the like(collectively referred to as “elements”). These elements may beimplemented using hardware, software, or combinations thereof. Whethersuch elements are implemented as hardware or software depends upon theparticular application and design constraints imposed on the overallsystem.

It should be noted that while aspects may be described below usingterminology commonly associated with 5G and later wireless technologies,aspects of the present disclosure can be applied in othergeneration-based communications systems, such as and including 3G and/or4G technologies.

As described, a machine learning model may be trained in a decentralizedmanner. The machine learning model may be an example of a deep neuralnetwork. In some examples, the machine learning model may be trained inmultiple edge devices, such as a mobile device, where local data samplesmay be specified as training data. The local data may be collected bythe mobile device via device sensors (e.g., positioning sensor,accelerometer, etc.), user input, and/or other methods of collectingdata. A system for training the machine learning model in adecentralized manner may be referred to as a federated learning system.

In some examples, the machine learning model may be trained to perform atask, such as typing assistance or vocabulary prediction. In suchexamples, local data samples may differ based on one or more attributesof the edge devices used for training the machine learning model. Theseattributes may include, but are not limited to, a geographic location, adefault language, or a user interface theme. In the present disclosure,edge devices used for training the machine learning model may bereferred to as participating devices or training devices. Aspects of thepresent disclosure are directed to grouping participating devices basedon shared attributes. Each group of participating devices may beassociated with a machine learning model of a zone. For example,participating devices may be grouped based on geographic location. Insuch an example, participating devices in Los Angeles may be groupedtogether and associated with a first zone and participating devices inNew York may be grouped together and associated with a second zone. Themachine learning model of a zone may be referred to as a zone model. Bygrouping participating devices based on inherent similarities ordifferences, aspects of the present disclosure may improve machinelearning models by providing customized machine learning models that arerelevant to a given group (e.g., zone).

As an example, a machine learning model may be trained to predict one ormore words based on a current word. The predicted words may improve auser's typing speed. In such an example, the machine learning model maylearn vocabulary over time. Some of the learned vocabulary may beassociated with a geographic location. Thus, a word used in one zone maynot be applicable to a word used in a different zone, even if both usersin both zones have the same language. Therefore, it may be desirable tocustomize machine learning models based on nuances associated with aparticular zone. In one example, a geographic location associated with afirst zone may refer to a beverage as “soda” and another geographiclocation associated with a second zone may refer to a beverage as “pop.”In this example, it may be desirable to customize a first zone model torecognize “soda” as a term for a beverage and also customize a secondzone model to recognize “pop” as a term for a beverage.

FIG. 1 is a diagram illustrating a network 100 in which aspects of thepresent disclosure may be practiced. The network 100 may be a 5G or NRnetwork or some other wireless network, such as an LTE network. Thewireless network 100 may include a number of BSs 110 (shown as BS 110 a,BS 110 b, BS 110 c, and BS 110 d) and other network entities. A BS is anentity that communicates with user equipment (UEs) and may also bereferred to as a base station, a NR BS, a Node B, a gNB, a 5G node B(NB), an access point, a transmit receive point (TRP), and/or the like.Each BS may provide communications coverage for a particular geographicarea. In 3GPP, the term “cell” can refer to a coverage area of a BSand/or a BS subsystem serving this coverage area, depending on thecontext in which the term is used.

A BS may provide communications coverage for a macro cell, a pico cell,a femto cell, and/or another type of cell. A macro cell may cover arelatively large geographic area (e.g., several kilometers in radius)and may allow unrestricted access by UEs with service subscription. Apico cell may cover a relatively small geographic area and may allowunrestricted access by UEs with service subscription. A femto cell maycover a relatively small geographic area (e.g., a home) and may allowrestricted access by UEs having association with the femto cell (e.g.,UEs in a closed subscriber group (CSG)). A BS for a macro cell may bereferred to as a macro BS. A BS for a pico cell may be referred to as apico BS. A BS for a femto cell may be referred to as a femto BS or ahome BS. In the example shown in FIG. 1 , a BS 110 a may be a macro BSfor a macro cell 102 a, a BS 110 b may be a pico BS for a pico cell 102b, and a BS 110 c may be a femto BS for a femto cell 102 c. A BS maysupport one or multiple (e.g., three) cells. The terms “eNB,” “basestation,” “NR BS,” “gNB,” “TRP,” “AP,” “node B,” “5G NB,” and “cell” maybe used interchangeably.

In some aspects, a cell may not necessarily be stationary, and thegeographic area of the cell may move according to the location of amobile BS. In some aspects, the BSs may be interconnected to one anotherand/or to one or more other BSs or network nodes (not shown) in thewireless network 100 through various types of backhaul interfaces suchas a direct physical connection, a virtual network, and/or the likeusing any suitable transport network.

The wireless network 100 may also include relay stations. A relaystation is an entity that can receive a transmission of data from anupstream station (e.g., a BS or a UE) and send a transmission of thedata to a downstream station (e.g., a UE or a BS). A relay station mayalso be a UE that can relay transmissions for other UEs. In the exampleshown in FIG. 1 , a relay station 110 d may communicate with macro BS110 a and a UE 120 d in order to facilitate communications between theBS 110 a and UE 120 d. A relay station may also be referred to as arelay BS, a relay base station, a relay, and/or the like.

The wireless network 100 may be a heterogeneous network that includesBSs of different types, e.g., macro BSs, pico BSs, femto BSs, relay BSs,and/or the like. These different types of BSs may have differenttransmit power levels, different coverage areas, and different impact oninterference in the wireless network 100. For example, macro BSs mayhave a high transmit power level (e.g., 5 to 40 Watts) whereas pico BSs,femto BSs, and relay BSs may have lower transmit power levels (e.g., 0.1to 2 Watts).

A network controller 130 may couple to a set of BSs and may providecoordination and control for these BSs. The network controller 130 maycommunicate with the BSs via a backhaul. The BSs may also communicatewith one another, e.g., directly or indirectly via a wireless orwireline backhaul.

UEs 120 (e.g., 120 a, 120 b, 120 c) may be dispersed throughout thewireless network 100, and each UE may be stationary or mobile. A UE mayalso be referred to as an access terminal, a terminal, a mobile station,a subscriber unit, a station, and/or the like. A UE may be a cellularphone (e.g., a smart phone), a personal digital assistant (PDA), awireless modem, a wireless communications device, a handheld device, alaptop computer, a cordless phone, a wireless local loop (WLL) station,a tablet, a camera, a gaming device, a netbook, a smartbook, anultrabook, a medical device or equipment, biometric sensors/devices,wearable devices (smart watches, smart clothing, smart glasses, smartwrist bands, smart jewelry (e.g., smart ring, smart bracelet)), anentertainment device (e.g., a music or video device, or a satelliteradio), a vehicular component or sensor, smart meters/sensors,industrial manufacturing equipment, a global positioning system device,or any other suitable device that is configured to communicate via awireless or wired medium.

Some UEs may be considered machine-type communications (MTC) or evolvedor enhanced machine-type communications (eMTC) UEs. MTC and eMTC UEsinclude, for example, robots, drones, remote devices, sensors, meters,monitors, location tags, and/or the like, that may communicate with abase station, another device (e.g., remote device), or some otherentity. A wireless node may provide, for example, connectivity for or toa network (e.g., a wide area network such as Internet or a cellularnetwork) via a wired or wireless communications link. Some UEs may beconsidered Internet-of-Things (IoT) devices, and/or may be implementedas NB-IoT (narrowband internet of things) devices. Some UEs may beconsidered a customer premises equipment (CPE). UE 120 may be includedinside a housing that houses components of UE 120, such as processorcomponents, memory components, and/or the like.

In general, any number of wireless networks may be deployed in a givengeographic area. Each wireless network may support a particular RAT andmay operate on one or more frequencies. A RAT may also be referred to asa radio technology, an air interface, and/or the like. A frequency mayalso be referred to as a carrier, a frequency channel, and/or the like.Each frequency may support a single RAT in a given geographic area inorder to avoid interference between wireless networks of different RATs.In some cases, NR or 5G RAT networks may be deployed.

In some aspects, two or more UEs 120 (e.g., shown as UE 120 a and UE 120e) may communicate directly using one or more sidelink channels (e.g.,without using a base station 110 as an intermediary to communicate withone another). For example, the UEs 120 may communicate usingpeer-to-peer (P2P) communications, device-to-device (D2D)communications, a vehicle-to-everything (V2X) protocol (e.g., which mayinclude a vehicle-to-vehicle (V2V) protocol, a vehicle-to-infrastructure(V2I) protocol, and/or the like), a mesh network, and/or the like. Inthis case, the UE 120 may perform scheduling operations, resourceselection operations, and/or other operations described elsewhere asbeing performed by the base station 110. For example, the base station110 may configure a UE 120 via downlink control information (DCI), radioresource control (RRC) signaling, a media access control-control element(MAC-CE) or via system information (e.g., a system information block(SIB).

In some implementations, a UE 120 may include a federated learning (FL)component 196 for receiving, at the UE 120 associated with a first groupof UEs, a first model from the base station 110 associated with a firstzone model of a number of zone models. In some examples, the first groupof UEs is associated with a first zone model, and a different group ofUEs is associated with each of the number of zone models. The methodfurther includes identifying, at the UE 120, a network device fortraining the first model based on one or both of a current connectivitystate of the UE 120 or a current resource use of the UE 120. The methodstill further includes transmitting, to the first network device, modelweight updates based on the training of the first model. The method alsoincludes receiving, from the base station 110, the first zone modelbased on the transmitted model weights updates. For illustrativepurposes, FIG. 1 only shows the FL component 196 of one UE 120 d. The FLcomponent 196 may be a component of each UE 120 in FIG. 1 .Additionally, the FL component 196 is not limited to UEs 120, othernetwork devices may include an FL component 196.

Additionally, in some implementations, a base station 110 may include afederated learning (FL) component 198 for receiving, at the base station110 associated with a first zone model of a number of zone models, aglobal model from a second network device associated with the globalmodel; transmitting, from the base station 110, the global model to UEs120 in a first group of UEs associated with a first zone model;receiving, at the base station 110, weights associated with the globalmodel from each UE 120 in the first group; updating, at the base station110, the first zone model based on the received weights; andtransmitting, from the base station 110, the updated first zone model toeach UE 120 in the first group. For illustrative purposes, FIG. 1 onlyshows the FL component 198 of one base station 110 a. The FL component198 may be a component of each base station 110 in FIG. 1 .Additionally, the FL component 198 is not limited to base stations 110,other network devices, such as an FL server (not shown in FIG. 1 ) or anetwork controller 130, may include an FL component 198.

As indicated above, FIG. 1 is provided merely as an example. Otherexamples may differ from what is described with regard to FIG. 1 .

FIG. 2 shows a block diagram of a design 200 of the base station 110 andUE 120, which may be one of the base stations and one of the UEs in FIG.1 . The base station 110 may be equipped with T antennas 234 a through234 t, and UE 120 may be equipped with R antennas 252 a through 252 r,where in general T≥1 and R≥1.

At the base station 110, a transmit processor 220 may receive data froma data source 212 for one or more UEs, select one or more modulation andcoding schemes (MCS) for each UE based at least in part on channelquality indicators (CQIs) received from the UE, process (e.g., encodeand modulate) the data for each UE based at least in part on the MCS(s)selected for the UE, and provide data symbols for all UEs. Decreasingthe MCS lowers throughput but increases reliability of the transmission.The transmit processor 220 may also process system information (e.g.,for semi-static resource partitioning information (SRPI) and/or thelike) and control information (e.g., CQI requests, grants, upper layersignaling, and/or the like) and provide overhead symbols and controlsymbols. The transmit processor 220 may also generate reference symbolsfor reference signals (e.g., the cell-specific reference signal (CRS))and synchronization signals (e.g., the primary synchronization signal(PSS) and secondary synchronization signal (SSS)). A transmit (TX)multiple-input multiple-output (MIMO) processor 230 may perform spatialprocessing (e.g., precoding) on the data symbols, the control symbols,the overhead symbols, and/or the reference symbols, if applicable, andmay provide T output symbol streams to T modulators (MODs) 232 a through232 t. Each modulator 232 may process a respective output symbol stream(e.g., for OFDM and/or the like) to obtain an output sample stream. Eachmodulator 232 may further process (e.g., convert to analog, amplify,filter, and upconvert) the output sample stream to obtain a downlinksignal. T downlink signals from modulators 232 a through 232 t may betransmitted via T antennas 234 a through 234 t, respectively. Accordingto various aspects described in more detail below, the synchronizationsignals can be generated with location encoding to convey additionalinformation.

At the UE 120, antennas 252 a through 252 r may receive the downlinksignals from the base station 110 and/or other base stations and mayprovide received signals to demodulators (DEMODs) 254 a through 254 r,respectively. Each demodulator 254 may condition (e.g., filter, amplify,downconvert, and digitize) a received signal to obtain input samples.Each demodulator 254 may further process the input samples (e.g., forOFDM and/or the like) to obtain received symbols. A MIMO detector 256may obtain received symbols from all R demodulators 254 a through 254 r,perform MIMO detection on the received symbols if applicable, andprovide detected symbols. A receive processor 258 may process (e.g.,demodulate and decode) the detected symbols, provide decoded data forthe UE 120 to a data sink 260, and provide decoded control informationand system information to a controller/processor 280. A channelprocessor may determine reference signal received power (RSRP), receivedsignal strength indicator (RSSI), reference signal received quality(RSRQ), channel quality indicator (CQI), and/or the like. In someaspects, one or more components of the UE 120 may be included in ahousing.

On the uplink, at the UE 120, a transmit processor 264 may receive andprocess data from a data source 262 and control information (e.g., forreports comprising RSRP, RSSI, RSRQ, CQI, and/or the like) from thecontroller/processor 280. Transmit processor 264 may also generatereference symbols for one or more reference signals. The symbols fromthe transmit processor 264 may be precoded by a TX MIMO processor 266 ifapplicable, further processed by modulators 254 a through 254 r (e.g.,for DFT-s-OFDM, CP-OFDM, and/or the like), and transmitted to the basestation 110. At the base station 110, the uplink signals from the UE 120and other UEs may be received by the antennas 234, processed by thedemodulators 254, detected by a MIMO detector 236 if applicable, andfurther processed by a receive processor 238 to obtain decoded data andcontrol information sent by the UE 120. The receive processor 238 mayprovide the decoded data to a data sink 239 and the decoded controlinformation to a controller/processor 240. The base station 110 mayinclude communications unit 244 and communicate to the networkcontroller 130 via the communications unit 244. The network controller130 may include a communications unit 294, a controller/processor 290,and a memory 292.

The controller/processor 240 of the base station 110, thecontroller/processor 280 of the UE 120, and/or any other component(s) ofFIG. 2 may perform one or more techniques associated with zone-basedfederated learning, as described in more detail elsewhere. For example,the controller/processor 240 of the base station 110, thecontroller/processor 280 of the UE 120, and/or any other component(s) ofFIG. 2 may perform or direct operations of, for example, the processesof FIGS. 6-8 and/or other processes as described. Memories 242 and 282may store data and program codes for the base station 110 and UE 120,respectively. A scheduler 246 may schedule UEs for data transmission onthe downlink and/or uplink.

In some aspects, the UE 120 may include means for receiving, at thefirst UE associated with a first group of UEs, a first model from afirst network device associated with the first zone model of a number ofzone models. In some examples, the first group of UEs is associated witha first zone model, and a different group of UEs is associated with eachof the number of zone models. The UE 120 further includes means foridentifying, at the first UE, a network device for training the firstmodel based on one or both of a current connectivity state of the UE ora current resource use of the UE. The UE 120 still further includesmeans for transmitting, to the first network device, model weightupdates based on the training of the first model. The UE 120 alsoincludes means for receiving, from the first network device, the firstzone model based on the transmitted model weights updates. Such meansmay include one or more components of the UE 120 described in connectionwith FIG. 2 .

In some aspects, the base station 110 may include means for receiving,at the first network device associated with a first zone model of anumber of zone models, a global model from a second network deviceassociated with the global model. The base station 110 further includesmeans for transmitting, from the first network device, the global modelto UEs in a first group of UEs associated with the first zone model. Insome examples, a different group of UEs may associated with each of thenumber of zone models. The base station 110 still further includes meansfor receiving, at the first network device, weights associated with theglobal model from each UE in the first group. The base station 110 alsoincludes means for updating, at the first network device, the first zonemodel based on the received weights. The base station 110 furtherincludes means for transmitting, from the first network device, theupdated first zone model to each UE in the first group. Such means mayinclude one or more components of the base station 110 described inconnection with FIG. 2 .

As indicated above, FIG. 2 is provided merely as an example. Otherexamples may differ from what is described with regard to FIG. 2 .

In some cases, different types of devices supporting different types ofapplications and/or services may coexist in a cell. Examples ofdifferent types of devices include UE handsets, customer premisesequipment (CPEs), vehicles, Internet of Things (IoT) devices, and/or thelike. Examples of different types of applications include ultra-reliablelow-latency communications (URLLC) applications, massive machine-typecommunications (mMTC) applications, enhanced mobile broadband (eMBB)applications, vehicle-to-anything (V2X) applications, and/or the like.Furthermore, in some cases, a single device may support differentapplications or services simultaneously.

FIG. 3 illustrates an example implementation of a system-on-a-chip (SOC)300, which may include a central processing unit (CPU) 302 or amulti-core CPU configured for zone-based federated learning, inaccordance with certain aspects of the present disclosure. The SOC 300may be included in the base station 110 or UE 120. Variables (e.g.,neural signals and synaptic weights), system parameters associated witha computational device (e.g., neural network with weights), delays,frequency bin information, and task information may be stored in amemory block associated with a neural processing unit (NPU) 308, in amemory block associated with a CPU 302, in a memory block associatedwith a graphics processing unit (GPU) 304, in a memory block associatedwith a digital signal processor (DSP) 306, in a memory block 318, or maybe distributed across multiple blocks. Instructions executed at the CPU302 may be loaded from a program memory associated with the CPU 302 ormay be loaded from a memory block 318.

The SOC 300 may also include additional processing blocks tailored tospecific functions, such as a GPU 304, a DSP 306, a connectivity block310, which may include fifth generation (5G) connectivity, fourthgeneration long term evolution (4G LTE) connectivity, Wi-Ficonnectivity, USB connectivity, Bluetooth connectivity, and the like,and a multimedia processor 312 that may, for example, detect andrecognize gestures. In one implementation, the NPU is implemented in theCPU, DSP, and/or GPU. The SOC 300 may also include a sensor processor314, image signal processors (ISPs) 316, and/or navigation module 320,which may include a global positioning system.

The SOC 300 may be based on an ARM instruction set. In an aspect of thepresent disclosure, the instructions loaded into the general-purposeprocessor 302 may comprise code to receive, at the first network deviceassociated with a first zone model of a number of zone models, a globalmodel from a second network device associated with the global model. Thegeneral-purpose processor 302 further includes program code to transmit,from the first network device, the global model to UEs in a first groupof UEs associated with the first zone model, a different group of UEsassociated with each of the number of zone models. The general-purposeprocessor 302 still further includes program code to receive, at thefirst network device, weights associated with the global model from eachUE in the first group. The general-purpose processor 302 also includesprogram code to update, at the first network device, the first zonemodel based on the received weights. The general-purpose processor 302further includes program code to transmit, from the first networkdevice, the updated first zone model to each UE in the first group.

In some aspects of the present disclosure, the instructions loaded intothe general-purpose processor 302 may comprise code to receive, at thefirst UE associated with a first group of UEs, a first model from afirst network device associated with the first zone model of a number ofzone models. In some examples, the first group of UEs is associated witha first zone model, and a different group of UEs are associated witheach of the number of zone models. The general-purpose processor 302further includes program code to identify, at the first UE, a networkdevice for training the first model based on one or both of a currentconnectivity state of the UE or a current resource use of the UE. Thegeneral-purpose processor 302 still further includes program code totransmit, to the first network device, model weight updates based on thetraining of the first model. The general-purpose processor 302 alsoincludes program code to receive, from the first network device, thefirst zone model based on the transmitted model weights updates.

Deep learning architectures may perform an object recognition task bylearning to represent inputs at successively higher levels ofabstraction in each layer, thereby building up a useful featurerepresentation of the input data. In this way, deep learning addresses amajor bottleneck of traditional machine learning. Prior to the advent ofdeep learning, a machine learning approach to an object recognitionproblem may have relied heavily on human engineered features, perhaps incombination with a shallow classifier. A shallow classifier may be atwo-class linear classifier, for example, in which a weighted sum of thefeature vector components may be compared with a threshold to predict towhich class the input belongs. Human engineered features may betemplates or kernels tailored to a specific problem domain by engineerswith domain expertise. Deep learning architectures, in contrast, maylearn to represent features that are similar to what a human engineermight design, but through training. Furthermore, a deep network maylearn to represent and recognize new types of features that a humanmight not have considered.

A deep learning architecture may learn a hierarchy of features. Ifpresented with visual data, for example, the first layer may learn torecognize relatively simple features, such as edges, in the inputstream. In another example, if presented with auditory data, the firstlayer may learn to recognize spectral power in specific frequencies. Thesecond layer, taking the output of the first layer as input, may learnto recognize combinations of features, such as simple shapes for visualdata or combinations of sounds for auditory data. For instance, higherlayers may learn to represent complex shapes in visual data or words inauditory data. Still higher layers may learn to recognize common visualobjects or spoken phrases.

Deep learning architectures may perform especially well when applied toproblems that have a natural hierarchical structure. For example, theclassification of motorized vehicles may benefit from first learning torecognize wheels, windshields, and other features. These features may becombined at higher layers in different ways to recognize cars, trucks,and airplanes.

Neural networks may be designed with a variety of connectivity patterns.In feed-forward networks, information is passed from lower to higherlayers, with each neuron in a given layer communicating to neurons inhigher layers. A hierarchical representation may be built up insuccessive layers of a feed-forward network, as described above. Neuralnetworks may also have recurrent or feedback (also called top-down)connections. In a recurrent connection, the output from a neuron in agiven layer may be communicated to another neuron in the same layer. Arecurrent architecture may be helpful in recognizing patterns that spanmore than one of the input data chunks that are delivered to the neuralnetwork in a sequence. A connection from a neuron in a given layer to aneuron in a lower layer is called a feedback (or top-down) connection. Anetwork with many feedback connections may be helpful when therecognition of a high-level concept may aid in discriminating theparticular low-level features of an input.

The connections between layers of a neural network may be fullyconnected or locally connected. FIG. 4A illustrates an example of afully connected neural network 402. In a fully connected neural network402, a neuron in a first layer may communicate its output to everyneuron in a second layer, so that each neuron in the second layer willreceive input from every neuron in the first layer. FIG. 4B illustratesan example of a locally connected neural network 404. In a locallyconnected neural network 404, a neuron in a first layer may be connectedto a limited number of neurons in the second layer. More generally, alocally connected layer of the locally connected neural network 404 maybe configured so that each neuron in a layer will have the same or asimilar connectivity pattern, but with connections strengths that mayhave different values (e.g., 410, 412, 414, and 416). The locallyconnected connectivity pattern may give rise to spatially distinctreceptive fields in a higher layer, because the higher layer neurons ina given region may receive inputs that are tuned through training to theproperties of a restricted portion of the total input to the network.

One example of a locally connected neural network is a convolutionalneural network. FIG. 4C illustrates an example of a convolutional neuralnetwork 406. The convolutional neural network 406 may be configured suchthat the connection strengths associated with the inputs for each neuronin the second layer are shared (e.g., 408). Convolutional neuralnetworks may be well suited to problems in which the spatial location ofinputs is meaningful.

One type of convolutional neural network is a deep convolutional network(DCN). FIG. 4D illustrates a detailed example of a DCN 400 designed torecognize visual features from an image 426 input from an imagecapturing device 430, such as a car-mounted camera. The DCN 400 of thecurrent example may be trained to identify traffic signs and a numberprovided on the traffic sign. Of course, the DCN 400 may be trained forother tasks, such as identifying lane markings or identifying trafficlights.

The DCN 400 may be trained with supervised learning. During training,the DCN 400 may be presented with an image, such as the image 426 of aspeed limit sign, and a forward pass may then be computed to produce anoutput 422. The DCN 400 may include a feature extraction section and aclassification section. Upon receiving the image 426, a convolutionallayer 432 may apply convolutional kernels (not shown) to the image 426to generate a first set of feature maps 418. As an example, theconvolutional kernel for the convolutional layer 432 may be a 5×5 kernelthat generates 28×28 feature maps. In the present example, because fourdifferent feature maps are generated in the first set of feature maps418, four different convolutional kernels were applied to the image 426at the convolutional layer 432. The convolutional kernels may also bereferred to as filters or convolutional filters.

The first set of feature maps 418 may be subsampled by a max poolinglayer (not shown) to generate a second set of feature maps 420. The maxpooling layer reduces the size of the first set of feature maps 418.That is, a size of the second set of feature maps 420, such as 14×14, isless than the size of the first set of feature maps 418, such as 28×28.The reduced size provides similar information to a subsequent layerwhile reducing memory consumption. The second set of feature maps 420may be further convolved via one or more subsequent convolutional layers(not shown) to generate one or more subsequent sets of feature maps (notshown).

In the example of FIG. 4D, the second set of feature maps 420 isconvolved to generate a first feature vector 424. Furthermore, the firstfeature vector 424 is further convolved to generate a second featurevector 428. Each feature of the second feature vector 428 may include anumber that corresponds to a possible feature of the image 426, such as“sign,” “60,” and “100.” A softmax function (not shown) may convert thenumbers in the second feature vector 428 to a probability. As such, anoutput 422 of the DCN 400 is a probability of the image 426 includingone or more features.

In the present example, the probabilities in the output 422 for “sign”and “60” are higher than the probabilities of the others of the output422, such as “30,” “40,” “50,” “70,” “80,” “90,” and “100”. Beforetraining, the output 422 produced by the DCN 400 is likely to beincorrect. Thus, an error may be calculated between the output 422 and atarget output. The target output is the ground truth of the image 426(e.g., “sign” and “60”). The weights of the DCN 400 may then be adjustedso the output 422 of the DCN 400 is more closely aligned with the targetoutput.

To adjust the weights, a learning algorithm may compute a gradientvector for the weights. The gradient may indicate an amount that anerror would increase or decrease if the weight were adjusted. At the toplayer, the gradient may correspond directly to the value of a weightconnecting an activated neuron in the penultimate layer and a neuron inthe output layer. In lower layers, the gradient may depend on the valueof the weights and on the computed error gradients of the higher layers.The weights may then be adjusted to reduce the error. This manner ofadjusting the weights may be referred to as “back propagation” as itinvolves a “backward pass” through the neural network.

In practice, the error gradient of weights may be calculated over asmall number of examples, so that the calculated gradient approximatesthe true error gradient. This approximation method may be referred to asstochastic gradient descent. Stochastic gradient descent may be repeateduntil the achievable error rate of the entire system has stoppeddecreasing or until the error rate has reached a target level. Afterlearning, the DCN may be presented with new images (e.g., the speedlimit sign of the image 426) and a forward pass through the network mayyield an output 422 that may be considered an inference or a predictionof the DCN.

Deep belief networks (DBNs) are probabilistic models comprising multiplelayers of hidden nodes. DBNs may be used to extract a hierarchicalrepresentation of training data sets. A DBN may be obtained by stackingup layers of Restricted Boltzmann Machines (RBMs). An RBM is a type ofartificial neural network that can learn a probability distribution overa set of inputs. Because RBMs can learn a probability distribution inthe absence of information about the class to which each input should becategorized, RBMs are often used in unsupervised learning. Using ahybrid unsupervised and supervised paradigm, the bottom RBMs of a DBNmay be trained in an unsupervised manner and may serve as featureextractors, and the top RBM may be trained in a supervised manner (on ajoint distribution of inputs from the previous layer and target classes)and may serve as a classifier.

Deep convolutional networks (DCNs) are networks of convolutionalnetworks, configured with additional pooling and normalization layers.DCNs have achieved state-of-the-art performance on many tasks. DCNs canbe trained using supervised learning in which both the input and outputtargets are known for many exemplars and are used to modify the weightsof the network by use of gradient descent methods.

DCNs may be feed-forward networks. In addition, as described above, theconnections from a neuron in a first layer of a DCN to a group ofneurons in the next higher layer are shared across the neurons in thefirst layer. The feed-forward and shared connections of DCNs may beexploited for fast processing. The computational burden of a DCN may bemuch less, for example, than that of a similarly sized neural networkthat comprises recurrent or feedback connections.

The processing of each layer of a convolutional network may beconsidered a spatially invariant template or basis projection. If theinput is first decomposed into multiple channels, such as the red,green, and blue channels of a color image, then the convolutionalnetwork trained on that input may be considered three-dimensional, withtwo spatial dimensions along the axes of the image and a third dimensioncapturing color information. The outputs of the convolutionalconnections may be considered to form a feature map in the subsequentlayer, with each element of the feature map (e.g., 220) receiving inputfrom a range of neurons in the previous layer (e.g., feature maps 218)and from each of the multiple channels. The values in the feature mapmay be further processed with a non-linearity, such as a rectification,max(0, x). Values from adjacent neurons may be further pooled, whichcorresponds to down sampling, and may provide additional localinvariance and dimensionality reduction. Normalization, whichcorresponds to whitening, may also be applied through lateral inhibitionbetween neurons in the feature map.

The performance of deep learning architectures may increase as morelabeled data points become available or as computational powerincreases. Modern deep neural networks are routinely trained withcomputing resources that are thousands of times greater than what wasavailable to a typical researcher just fifteen years ago. Newarchitectures and training paradigms may further boost the performanceof deep learning. Rectified linear units may reduce a training issueknown as vanishing gradients. New training techniques may reduceover-fitting and thus enable larger models to achieve bettergeneralization. Encapsulation techniques may abstract data in a givenreceptive field and further boost overall performance.

FIG. 5 is a block diagram illustrating a deep convolutional network 550.The deep convolutional network 550 may include multiple different typesof layers based on connectivity and weight sharing. As shown in FIG. 5 ,the deep convolutional network 550 includes the convolution blocks 554A,554B. Each of the convolution blocks 554A, 554B may be configured with aconvolution layer (CONV) 356, a normalization layer (LNorm) 558, and amax pooling layer (MAX POOL) 560.

The convolution layers 556 may include one or more convolutionalfilters, which may be applied to the input data to generate a featuremap. Although only two of the convolution blocks 554A, 554B are shown,the present disclosure is not so limiting, and instead, any number ofthe convolution blocks 554A, 554B may be included in the deepconvolutional network 550 according to design preference. Thenormalization layer 558 may normalize the output of the convolutionfilters. For example, the normalization layer 558 may provide whiteningor lateral inhibition. The max pooling layer 560 may provide downsampling aggregation over space for local invariance and dimensionalityreduction.

The parallel filter banks, for example, of a deep convolutional networkmay be loaded on a CPU 302 or GPU 304 of an SOC 300 to achieve highperformance and low power consumption. In alternative embodiments, theparallel filter banks may be loaded on the DSP 306 or an ISP 316 of anSOC 300. In addition, the deep convolutional network 550 may accessother processing blocks that may be present on the SOC 300, such assensor processor 314 and navigation module 320, dedicated, respectively,to sensors and navigation.

The deep convolutional network 550 may also include one or more fullyconnected layers 562 (FC1 and FC2). The deep convolutional network 550may further include a logistic regression (LR) layer 564. Between eachlayer 556, 558, 560, 562, 564 of the deep convolutional network 550 areweights (not shown) that are to be updated. The output of each of thelayers (e.g., 556, 558, 560, 562, 564) may serve as an input of asucceeding one of the layers (e.g., 556, 558, 560, 562, 564) in the deepconvolutional network 550 to learn hierarchical feature representationsfrom input data 552 (e.g., images, audio, video, sensor data and/orother input data) supplied at the first of the convolution blocks 554A.The output of the deep convolutional network 550 is a classificationscore 566 for the input data 552. The classification score 566 may be aset of probabilities, where each probability is the probability of theinput data, including a feature from a set of features.

As indicated above, FIGS. 3-5 are provided as examples. Other examplesmay differ from what is described with respect to FIGS. 3-5 .

As described, aspects of the present disclosure are directed to groupingparticipating devices based on shared attributes. For example,participating devices may be grouped based on geographic location. Eachgroup of participating devices may be associated with a machine learningmodel of a zone. In such an example, participating devices in LosAngeles may be grouped together and associated with a first zone andparticipating devices in New York may be grouped together and associatedwith a second zone. The machine learning model of a zone may be referredto as a zone model. By grouping participating devices based on inherentsimilarities or differences, aspects of the present disclosure mayimprove machine learning models by providing customized machine learningmodels that are relevant to a given group (e.g., zone).

FIG. 6 is a diagram illustrating an example 600 of different zones in afederated learning system, in accordance with aspects of the presentdisclosure. In the example 600 of FIG. 6 , each UE 620 may be an exampleof a device participating in federated learning. Such devices may bereferred to as participating devices. Additionally, each UE 620 may bean example of a UE 120 as described with reference to FIGS. 1 and 2 . Insome implementations, as shown in the example 600 of FIG. 6 , each UE620 may be placed in a group 610, 612, 614 based on one or more commonattributes or settings. Each group 610, 612, 614 may correspond to aparticular zone. For example, as shown in FIG. 6 , a first group 610corresponds to a first zone, a second group 612 corresponds to a secondzone, and a third group 614 corresponds to a third zone. In someexamples, a UE 620 may be placed in more than one group 610, 612, 614(not shown in FIG. 6 ). Additionally, or alternatively, two or morezones may overlap (not shown in FIG. 6 ). As described, the attributesand settings may include, but are not limited to, a geographic location,a default language, or a user interface theme. As an example, each group610, 612, 614 may be based on a UE's geographic location. In thisexample, the UEs 620 in a first group 610 have a common geographiclocation, the UEs 620 in a second group 612 have a common geographiclocation, and the UEs 620 in a third group 614 have a common geographiclocation. Additionally, as shown in FIG. 6 , each group 610, 612, 614may be associated with a different zone server 654, 656, 658, where eachzone server 654, 656, 658 stores a different zone model 604, 606, 608. Azone server may also be referred to as a zone device. The zone models604, 606, 606 may be examples of machine learning models, such as deepneural networks. Each zone server 654, 656, 658 may be a differentnetwork device, such as a federated learning (FL) server. In someexamples, each zone server 654, 656, 658 may be integrated with a basestation, such as a base station 110 of FIGS. 1 and 2 , or a backendserver device. As described, each zone model 604, 606, 608 may becustomized based on training performed at the participating devices of acorresponding group 610, 612, 614. Furthermore, as shown in FIG. 6 ,each zone model 604, 606, 608 may be associated with a global model 602stored in a global server 652, such as a network device (e.g., server).The global server 652 may be in a same location as one or more of thezone server 654, 656, 658. Alternatively, each device 652, 654, 656, 658may be in a different geographic location. As described, each zone model604, 606, 608 may be customized based on training performed at UEs 620of an associated group.

FIG. 7 is a timing diagram illustrating an example 700 of customizingzone models 604, 606, 608, in accordance with aspects of the presentdisclosure. In the example of FIG. 7 , prior to training, UEs 620 (notshown in FIG. 7 ) are grouped in either a first group 610 associatedwith a first zone model 604 or a second group 612 associated with asecond zone model 606. The UEs 620 may be grouped based on one or morecommon attributes. As shown in FIG. 7 , at time t1, the global server652 transmits the global model 602 to the first zone server 654 and thesecond zone server 656. The global model 602 may be stored at each zoneserver 654, 656. In the current example, the global model 602transmitted at time t1 may be an untrained global model 602 (e.g., abare global model 602). Additionally, at time t2, the first zone server654 and the second zone server 656 transmit the global model 602 to theUEs 620 in the first group 610 and the second group 612, respectively.

After receiving the global model 602, at time t3, each UE 620 in thefirst group 610 and the second group 612 individually trains the globalmodel 602 based on local data. At time t4, the respective UEs 620 in thefirst group 610 and the second group 612 transmit weight updates to thefirst zone server 654 and the second zone server 656, respectively. Eachweight update may be based on the training of the global model performedat the respective UEs 620. In some implementations, the weight updatemay be a difference between an original weight of a global model (e.g.,a weight prior to training) and a weight that is updated based on thetraining. The weight update may include one or more updated weights orweight deltas. That is, in some examples, the weight update does nottransmit all weights of a model. At time t5, the first zone server 654and the second zone server 656 may update the stored model (e.g., theglobal mode 602 received at time t1) based on the weight updatesreceived from the respective UEs 620 of the associated groups 610, 612.In the example of FIG. 7 , respective zone models 604, 606 may begenerated based the update to the stored model, such as the global model602. For ease of explanation, the process at time t5 refers to updatingthe stored model. In some examples, at an initial training period, theglobal model 602 and the zone models 604, 606 are the same. However,after the first training round each zone server 654, 656 may update astored model based on weight updates reported by associated UEs.Updating the stored model creates the zone model 604, 606 for subsequenttraining rounds.

In some implementations, the received weight updates may be aggregatedbefore updating the global model 602. For example, the weight updatesmay be averaged and the stored global model 602 may be updated based onthe averaged weight update. In some examples, each update may customizethe global model 602 for UEs 620 associated with a specific zone. As anexample, the global model 602 stored at the first zone server 654 may becustomized to UEs 620 of the first group 610, such that the updatedglobal model 602 may be referred to as a first zone model 604. Asanother example, the global model 602 stored at the second zone server656 may be customized to UEs 620 of the second group 612, such that theupdated global model 602 may be referred to as a second zone model 606.

Additionally, in some implementations, at time t6, each zone server 654,656 transmits the respective zone model 604, 606 to UEs 620 of eachassociated group 610, 612. Each UE 620 may use the respective zone model604, 606 to perform one or more tasks for an application, such asvocabulary prediction, based on locally collected data. Additionally,each UE 620 may continue to train the respective zone model 604, 606with locally collected data. That is, each UE 620 may train therespective zone model 604, 606 while also using the respective zonemodel 604, 606 for a corresponding task, such as inference orprediction. Furthermore, as shown in FIG. 7 , at time t7, each zoneserver 654, 656 transmits the respective weight updates to the globalserver 652. The weight updates transmitted at time t7 may be raw weightupdates or aggregated weight updates, such as an averaged weightupdates. Raw weight updates may be an example of weight updates receivedat a respective zone server 654, 656 prior to aggregation by therespective zone server 654, 656. At time t8, the global server 652 mayupdate the global model 602 based on the received weight updates. Insome implementations, the global server 652 may average the receivedweight updates and update the global model 602 based on the average ofthe received weight updates. In such implementations, the global server652 may generate a global averaged model based on the update to theglobal model 602. In some examples, the global averaged model may beperiodically transmitted to the zone level devices 654, 656. In suchexamples, the global averaged model may improve performance of a certainzone if a performance of the certain zone is less than a performance ofother zones. In some examples, the performance may be based onclassification accuracy, classification speed, and/or other performancemetrics. The global averaged model may refresh a zone model of thecertain zone to improve performance. Additionally, or alternatively, theglobal averaged model may initiate a new zone. In some implementations,the process described from times t1 to t8 may repeat for each UE 620until training termination. The training termination may be caused by aUE 620 leaving a group 610, 612, a UE 620 removing an applicationassociated with a model 604, 606, or other factors.

As an example, as shown in FIG. 7 , at time t9, a third zone server 658and third group 614 may be added as a new zone (e.g., a third zone). Inthis example, a third zone model associated with the third zone server658 and third group 614 may not have a same number of trainingiterations in comparison to zone models of other zones, such as a firstzone and a second zone, because UEs 620 associated with the third group614 have yet to train the third zone model. Thus, at time t10, toimprove overall performance and reduce training time, the global server652 may transmit the updated global model 602 to the third zone server658, such that the third zone model associated with the third zoneserver 658 may be initiated based on training performed by UEs 620associated with other groups 610, 612. In other implementations, ratherthan transmitting the updated global model 602, the updated global model602 may transmit weights from another zone, such as the first zone. Insuch implementations, the weights from a pre-existing zone may betransmitted to the new zone, such as the third zone associated with thethird zone server 658, based on the new zone and the pre-existing zonehaving one or more common attributes. For example, UEs 620 associatedwith groups of the pre-existing zone and the new zone may share a samedefault language or the UEs 620 may be in a similar geographic region,such as Western United States. As shown in FIG. 7 , at time t11, thethird zone server 658 transmits the updated global model 602 to each UE620 in the third group 614. The UEs 620 in the third group 614 may trainthe model 602 as described above with reference to time t3. The processfor the UEs 620 of the third group 614, the third zone server 658, andthe global server 652 may continue as described above with reference totimes t4-t8. Additionally, as described, the process described fromtimes t1 to t8 may repeat for each UE 620 in the third group 614 untiltraining is terminated. In some examples, training may be terminatedwhen a desired accuracy is achieved. In the example of FIG. 7 , thetraining may be performed on a subset of participating devices in eachzone. In some examples, the training may be performed in parallel ateach participating device in the zone. Additionally, the averaging ofthe weights may be performed when all weights have been collected at azone server from the different respective participating devices. Furtheraveraging/aggregation can be performed at the global level where thezone based devices act as participants.

In some implementations, a participating device, such as a UE, may beexcluded from federated learning training for a duration of time inresponse to the participating device migrating between zones. As anexample, a UE may move from a first zone to a second zone. In such anexample, based on the UE moving from the first zone to the second zone,the UE may be excluded from training a second zone model associated withthe second zone for a period of time.

In some implementations, a new zone, such as a migration zone, may begenerated for participating devices moving between a set of zones. Thenew zone may be customized based on habits of the participating devicemoving between the set of zones. As an example, a set of participatingdevices may move between a zone associated with a suburb to a zoneassociated with a city center. In such an example, habits andpreferences of participating devices may differ based on whether aparticipating device stays in the suburb, stays in the city center, ormoves between the suburb and the city center. Thus, a new zone, such asa migration zone, may cater to the habits and preferences of the set ofparticipating devices that move between the zone associated with thesuburb and the zone associated with the city center.

Aspects of the present disclosure are not limited to training onefederated learning model, such as a zone model, at a participatingdevice. In some implementations, a participating device may participatein training multiple federated learning models. In such implementations,the participating devices may include a federated learning manager tomanage device resources such that device resources may be used in anoptimal manner. The federated learning manager may be middleware storedand executed on each participating device.

FIG. 8 is a block diagram illustrating an example of a participatingdevice 800 including a federated learning (FL) manager 802, inaccordance with aspects of the present disclosure. In the example ofFIG. 8 , the participating device 800 may be an example of a UE, such asa UE 120 or 620 as described with reference to FIGS. 1, 2, 6, and 7 ,respectively. Additionally, zone servers 850 may be examples of the zoneservers 654, 656, 658 as described with reference to FIGS. 6 and 7 . Asshown in FIG. 8 , the participating device 800 may include multiplecomponents, such as a local weight storage 804, a global weight storage814, a model storage 806, a model runner 816, a processed data storage808, a data preprocessor 810, a raw data storage 812, an inter-processcommunication component 818, and a data collector 822. The variousstorage components 804, 806, 808, 812, 814 may be different partitionsor storage locations in a same storage device, such as the memory 282 asdescribed with reference to FIG. 2 . In another example, the storagecomponents 804, 806, 808, 812, 814 may be different storage devices. Aninter-process communication component 818, such as a bus or acontroller/processor, may facilitate communication between the differentcomponents 804, 806, 808, 810, 812, 814, 816. The inter-processcommunication component 818 may be an example of thecontroller/processor 280 as described with reference to FIG. 2 .Additionally, as shown in FIG. 8 , the participating device may includeapplications 820. The applications 820 may be stored in a storagecomponent 804, 806, 808, 812, 814, or another storage location (notshown in FIG. 8 ). In some examples, the applications 820 may use aninterface of the participating device 800.

In some examples, an FL manager 802 controls data collection using oneor more data collectors 822. Each data collector 822 may collect datafrom a sensor (not shown in FIG. 8 ) at a sampling rate. In someimplementations, a data collector 822 may be embedded with another datacollector 822, such that both data collectors 822 simultaneously collectdifferent types of data. Controlling the data collection via the FLmanager 802 may improve resource use, such as battery use and/orprocessor use, because the FL manager 802 may prevent multiple datacollectors 822 from collecting the same data. Additionally, sensoraccess control may be simplified based on the FL manager 802 controllingthe data collection. In some examples, the FL manager 802 maydynamically (e.g., on-demand) configure one or more of sensor types,sampling rates, and a period for flushing data from memory (not shown inFIG. 8 ) to storage, such as processed data storage 808. Each model mayinform the FL manager 802 of the type of data it needs for training anda specified sampling rate. Based on the information provided by eachmodel, the FL manager 802 may identify the appropriate data collectors822 to invoke and a corresponding sampling rate. In someimplementations, the FL manager 802 may use one or more policies tobalance sensing accuracy (e.g., a sampling rate) with resourceconsumption (e.g., battery use, process load, etc.).

In the example of FIG. 8 , the data collectors 822 store data obtainedfrom one or more sensors (not shown in FIG. 8 ) in the raw data storage812. Additionally, the data collectors 822 may inform the FL manager 802when new data is added to the raw data storage 812. In some examples,the data collectors 822 may buffer a certain amount of sensed data inmemory before committing the sensed data to the raw data storage 812.The FL manager 802 may dynamically reconfigure the data flushing periodthat defines when the data is written to the raw data storage 812. Insuch examples, the data flushing period may be initial set by the datacollectors 822.

In some examples, a model may use the raw data. In other examples, amodel may specify additional processing for the raw data. The additionalprocessing may be performed by a data processor 810. Although not shownin FIG. 8 , the phone 800 may include one or more data processors 810.Additionally, one or more data processors 810 may be model-specific. Insome examples, the FL manager 802 may determine when to invoke themodel-specific data processors 810. Each data processor 810 may storedata in the processed data storage 808. The data may be stored at aninterval or based on new data being available in the raw data storage812. In some examples, all data is pre-processed before initiating a newlocal model training operation.

In some examples, the data processor 810 and data collectors 822 may beimplemented by third-party developers. In some such examples, the FLmanager 802 may use an inter-process communication (IPC) 818 functionprovided by the phone's 800 operating system to interact withthird-party components.

As described, the FL manager 802 may initiate a model trainer for agiven model and determines a location of the data in the processed datastorage 808 or raw data storage 812. After the training is completed,the model trainer may store the newly computed weights in the localweight storage 804. Additionally, the FL manager 802 may determine whenthe stored weights may be uploaded to a network device.

In some examples, the FL manager 802 may receive multiple models fromone or more zone servers 850. That is, multiple models (e.g., federatedlearning models or applications) may be provided to the participatingdevice 800. As an example, a first application may be a text predictionmodel and a second application may be a location based advertisingmodel. In such examples, the FL manager 802 may determine a trainingtime for each model. In some examples, the participating device 800 maybe associated with two different zone servers, where each zone server isassociated with a different zone. Each zone server may transmit adifferent model. As another example, a single zone server may transmittwo or more different zone models.

The models may be stored in the model storage 806. Local weights of eachmodel may be stored in local weight storage 804 and global weights maybe stored in the global weight storage 814. In some implementations, theFL manager 802 may work in conjunction with one or more components 804,806, 808, 810, 812, 814, 816 of the participating device 800 todetermine a training priority of the various models stored in the modelstorage 806. In some examples, a priority of the model may be determinedbased on various criteria, such as, but not limited to, one or more of anumber of samples available for training for a given model, a currentaccuracy of the model, an estimated model training time determined basedon previous training times, and whether the training can be successfullycompleted based on current resources availability (e.g., battery levels,current system load, etc.). Additionally, the FL manager 802 may managea local training state of the various models stored in the model storage806. As an example, the FL manager 802 may stop training a first modeland start training a second model. In such an example, the FL manager802 may store the local weights of the first model in the local weightstorage 804 to maintain the training state of the first model, such thatthe training may resume at a later time.

In some implementations, the FL manager 802 may determine current deviceresources to assess whether one or more models may be locally trained(e.g., trained on-device). It may be desirable to locally train themodel to preserve data privacy. Still, local training may be limitedbecause the participating device 800, such as UEs and edge-devices, mayhave a limited amount of resources. In such implementations, the FLmanager 802 may use a training proxy if the current device resourcessatisfy a resource condition and a current connectivity state satisfiesa connection condition.

As described, an amount of available resources, such as available memoryor processer load, may prevent the participating device 800 from locallytraining a model. In this example, the resource condition may besatisfied when an amount of available resources prevents local training.That is, the amount of available resources may be less than a threshold.In some examples, the FL manager 802 may determine the currentconnectivity state when the resource condition is satisfied. Theconnectivity state refers to a connection status between theparticipating device 800 and a network device over a communicationchannel, such as Wi-Fi channel or a cellular channel. In such anexample, the connection condition may be satisfied if the participatingdevice can communicate with a network device, such as an inter-networkor intra-network device, over a communication channel. In this example,the FL manager 802 may use the network device as a proxy for trainingthe model.

In some implementations, a training proxy may be individually controlledby each participating device to improve training speed while stillpreserving privacy. The training proxy may be a network device that mayreceive both a model and training data. The network device may train themodel and return the trained weights and biases to the participatingdevice. In some examples, the training proxy may delete datacorresponding to the model, weights, and biases after the trainingsession. Furthermore, in some examples, the training proxy may notunderstand an overall context of the model. Rather, the training proxymay only be responsible for training the model. Additionally, a globalserver may be unaware of the training proxy. Because of thedecentralized nature of training, and because the training proxy isunaware of the overall context, the privacy of the participating devicemay be preserved.

FIG. 9 is a timing diagram illustrating an example 900 of federatedlearning with a training proxy, in accordance with aspects of thepresent disclosure. The example 900 includes a zone server 902, a UE620, and a proxy device 904. The zone server 902 may be an example of azone server 654, 656, 658 as described with reference to FIGS. 6 and 7 .The UE 620 may be an example of a UE 120 as described with reference toFIGS. 1 and 2 . The UE 620 may include a federated learning (FL) manager802 as described with reference to FIG. 8 . The proxy device 904 may bea wired or wireless network device that shares a network with the UE620. In such an example, the proxy device 904 may be an intra-netdevice. Alternatively, the proxy device 904 may be a cloud device, suchthat the proxy device 904 and the UE 620 are geographically separated.

As shown in FIG. 9 , at time t1, the zone server 902 transmits a modelto a UE 620. The model may be a bare global model, a global averagedmodel, or a zone model. Additionally, the model may be one of multiplemodels stored at the UE 620 for federated learning. At time t2, based onreceiving the model, the FL manager 802 may determine whether a localtraining condition is satisfied. As described, the local trainingcondition may be satisfied if the available resources at the UE 620support local training. That is, the UE 620 may be capable of localtraining if an amount of available resources is greater than athreshold. Alternatively, the UE 620 may not be capable of localtraining if an amount of available resources is less than a threshold.For exemplary purposes, the example of FIG. 9 assumes the UE 620 is notcapable of locally training the received model.

Based on determining the local training condition is not satisfied, attime t3, the FL manager 802 determines if a connection condition issatisfied. As described, the connection condition may be satisfied ifthe UE 620 may communicate with the proxy device 904 via a communicationchannel, such as a cellular channel, Wi-Fi channel, or other type ofcommunication channel. In the example of FIG. 9 , it is assumed thecommunication condition is satisfied. Based on determining thecommunication condition is satisfied, at time t4, the UE 620 maytransmit the model and local training data to the proxy device 904. Insome implementations, the model and the local training data may betransmitted via a secure connection. At time t5, the proxy device 904may generate a secure container for training the model based on thelocal training data. After training the model, the proxy device maytransmit the model weights and the local training data to the UE 620(time t6). In some examples, the proxy device may also transmit metadata for the UE to identify the model that was trained. After trainingthe model, the proxy device 904 may delete stored data, such as themodel and/or training, associated with the training. The data may bestored to improve data privacy. At time t7, the UE 620 may transmitupdated weights corresponding to the trained model to the zone server902. In the example of FIG. 9 , the zone server 902 may be unaware ofthe use of the proxy device 904 for training the model. Additionally,the proxy device 904 may be unaware of a global context of the model.Therefore, the privacy of local data collected by the UE 620 fortraining may be preserved.

FIG. 10 is a flow diagram illustrating an example process 100 formanaging model updates by a first network device, in accordance withaspects of the present disclosure is a diagram illustrating an exampleprocess 1000 performed, for example, by a network device, in accordancewith various aspects of the present disclosure. In some implementations,the process 1000 may be performed by a wireless communication deviceoperating as or within a network device, such as one of the basestations 110 described above with respect to FIGS. 1 and 2 , or a zoneserver, such as one of the zone servers 654, 656, 658, 902 describedabove with respect to FIGS. 6, 7, and 9 , respectively.

As shown in FIG. 10 , at block 1002, the process 1000 receives, at thefirst network device associated with a first zone model of a number ofzone models, a global model from a second network device associated withthe global model. Each zone model may be associated with a differentzone. As an example, the first zone model may be associated with a firstzone. The first network device may be an example of a zone server andthe second network device may be an example of a global server. Theglobal model may be an example of the global model 602 described withreference to FIGS. 6, 7, and 9 . At block 1004, the process 1000transmits, from the first network device, the global model to UEs in afirst group of UEs associated with a first zone. In some examples, adifferent group of UEs is associated with each of the number of zonemodels. As an example, UEs may be grouped based on one or moreattributes. At block 1006, the process 1000 receives, at the firstnetwork device, weights associated with the global model from each UE inthe first group. In some examples, the weights associated with theglobal model may be a set of weight deltas for one or more weights ofthe model. In some such examples, the UE does not transmit the entireweights set. Rather, the UE may transmit a change between an originalweight and a trained weight. At block 1008, the process 1000 updates atthe first network device, the first zone model based on the receivedweights. At block 1010, the process 1000 transmits, from the firstnetwork device, the updated first zone model to each UE in the firstgroup.

FIG. 11 is a flow diagram illustrating an example process 1100 fortraining models by a UE, in accordance with aspects of the presentdisclosure is a diagram illustrating an example process 1100 performed,for example, by a network device, in accordance with various aspects ofthe present disclosure. In some implementations, the process 1100 may beperformed by a network device operating as or within a UE, such as oneof the UEs 120, 620, 800, described above with respect to FIGS. 1, 2, 6,7, 8, and 9 , respectively.

As shown in FIG. 11 , at block 1102, the process 1100 receives, at thefirst UE associated with a first group of UEs, a first model from afirst network device associated with a first zone model of a number ofzone models. In some examples, the first model may be an example of aglobal model. In such examples, the first UE receives the global modelprior to an initial training phase. In some examples, the first group ofUEs is associated with the first zone model, and a different group ofUEs are associated with each of the number of zone models. At block1104, the process 1100 identifies, at the first UE, a network device fortraining the first model based on one or both of a current connectivitystate of the UE or a current resource use of the UE. In some examples,the network device is the first UE as described with reference to FIGS.6 and 7 . In some other examples, the network device is a proxy asdescribed with reference to FIG. 9 . At block 1106, the process 1100transmits, to the first network device, model weight updates based onthe training of the first model. In some examples, the model weightupdates may be a set of weight deltas for one or more weights of themodel. In some such examples, the UE does not transmit the entireweights set. At block 1108, the process 1100 receives, from the firstnetwork device, the first zone model based on the transmitted modelweights updates. In some examples, such as the example described withreference to FIG. 7 , a zone server updates a global model based on themodel weight updates to generate a first zone model.

Support for multiple dependent claims will be added after initial reviewof the application

Implementation examples are described in the following numbered clauses:

-   -   1. A method for managing model updates by a first network        device, comprising:        -   receiving, at the first network device associated with a            first zone model of a plurality of zone models, a global            model from a second network device associated with the            global model;        -   transmitting, from the first network device, the global            model to user equipment (UEs) in a first group of UEs            associated with the first zone model, a different group of            UEs associated with each of the plurality of zone models;        -   receiving, at the first network device, weights associated            with the global model from each UE in the first group;        -   updating, at the first network device, the first zone model            based on the received weights; and        -   transmitting, from the first network device, the updated            first zone model to each UE in the first group.    -   2. The method of Clause 1, in which the global model is an        untrained global model.    -   3. The method of Clause 1, in which the global model comprises        weight updates from one or more zone models of the plurality of        zone models.    -   4. The method of Clause 1, in which:        -   the global model is a second zone model generated at a third            network device associated with the second zone model; and        -   one or more attributes of the first zone model and the            second zone model satisfy a similarity condition.    -   5. The method of any of Clauses 1-4, in which UEs in the first        group of UEs are grouped based on one or more attributes.    -   6. The method of Clause 5, in which the one or more attributes        comprise one or more of a geographic location, a default        language, or a user interface theme.    -   7. The method of any of Clauses 1-6, further comprising        excluding a UE in the first group of UEs for a period of time        based on the UE moving to the first group from a second group of        UEs.    -   8. The method of any of Clause 1-7, in which the first group of        UEs comprises UEs that change between a first attribute        corresponding to a second group of UE and a second attribute        corresponding to a third group of UEs.    -   9. The method of any of Clause 1-8, further comprising:        -   averaging the weights received from each UE in the first            group; and        -   transmitting an average of the weights to the second network            device.    -   10. The method of Clause 9, in which updating the first zone        model comprises updating the first zone model based on the        average of the weights.    -   11. The method of Clause 9, further comprising receiving a        global averaged model based on transmitting the average of the        weights to the second network device.    -   12. A method performed by a first user equipment (UE),        comprising:        -   receiving, at the first UE associated with a first group of            UEs, a first model from a first network device associated            with the first zone model of a plurality of zone models, the            first group of UEs associated with a first zone model, and a            different group of UEs associated with each of the plurality            of zone models;        -   identifying, at the first UE, a network device for training            the first model based on one or both of a current            connectivity state of the UE or a current resource use of            the UE;        -   transmitting, to the first network device, model weight            updates based on the training of the first model; and        -   receiving, from the first network device, the first zone            model based on the transmitted model weights updates.    -   13. The method of Clause 12, in which identifying the network        device comprises identifying the first UE as the network device        based on the current connectivity state of the UE satisfying a        connectivity condition and the current resource use satisfying a        resource condition, and the method further comprises:        -   training the model at the first UE based on local training            data collected at the first UE based on identifying the UE            as the network device; and        -   generating the model weight updates based on the training.    -   14. The method of any of Clauses 12-13, further comprising        maintaining a local training state of the model across training        iterations of the model.    -   15. The method of any of Clauses 12-13, further comprising:        -   receiving a set of models at the first UE, the first model            being one model of the set of models; and        -   training the model from of the set of models based on a            training priority of the model being higher than a training            priority associated with each other model of the set of            models.    -   16. The method of Clause 12, in which identifying the network        device comprises identifying a proxy device as the network        device based on one or both of the current connectivity state of        the UE failing to satisfy a connectivity condition or the        current resource use failing to satisfy a resource condition,        and the method further comprises:        -   transmitting the model and training data collected at the            first UE to the proxy device for training the model at the            proxy device using the training data collected at the first            UE; and        -   receiving the model weights updates from the proxy device            based on the training performed at the proxy device.    -   17. The method of Clause 16, in which the proxy device and the        UE are intra-network devices or inter-network devices.    -   18. The method of any of Clauses 12-17, in which the model is an        untrained model.    -   19. The method of any of Clauses 12-17, in which the model        comprises weight updates from one or more zone models of the        plurality of zone models.    -   20. The method of any of Clauses 12-17, in which:        -   the model is a second zone model generated at a third            network device associated with the second zone model; and        -   one or more attributes of the first zone model and the            second zone model satisfy a similarity condition.    -   21. The method of any of Clause 12-20, in which one or more        attributes of the first UE are the same as one or more        attributes of each respective UE in the first group of UEs.    -   22. The method of Clause 21, in which the one or more attributes        comprise one or more of a geographic location, a default        language, or a user interface theme.

The foregoing disclosure provides illustration and description, but isnot intended to be exhaustive or to limit the aspects to the preciseform disclosed. Modifications and variations may be made in light of theabove disclosure or may be acquired from practice of the aspects.

As used, the term “component” is intended to be broadly construed ashardware, firmware, and/or a combination of hardware and software. Asused, a processor is implemented in hardware, firmware, and/or acombination of hardware and software.

Some aspects are described in connection with thresholds. As used,satisfying a threshold may, depending on the context, refer to a valuebeing greater than the threshold, greater than or equal to thethreshold, less than the threshold, less than or equal to the threshold,equal to the threshold, not equal to the threshold, and/or the like.

It will be apparent that systems and/or methods described may beimplemented in different forms of hardware, firmware, and/or acombination of hardware and software. The actual specialized controlhardware or software code used to implement these systems and/or methodsis not limiting of the aspects. Thus, the operation and behavior of thesystems and/or methods were described without reference to specificsoftware code—it being understood that software and hardware can bedesigned to implement the systems and/or methods based, at least inpart, on the description.

Even though particular combinations of features are recited in theclaims and/or disclosed in the specification, these combinations are notintended to limit the disclosure of various aspects. In fact, many ofthese features may be combined in ways not specifically recited in theclaims and/or disclosed in the specification. Although each dependentclaim listed below may directly depend on only one claim, the disclosureof various aspects includes each dependent claim in combination withevery other claim in the claim set. A phrase referring to “at least oneof” a list of items refers to any combination of those items, includingsingle members. As an example, “at least one of: a, b, or c” is intendedto cover a, b, c, a-b, a-c, b-c, and a-b-c, as well as any combinationwith multiples of the same element (e.g., a-a, a-a-a, a-a-b, a-a-c,a-b-b, a-c-c, b-b, b-b-b, b-b-c, c-c, and c-c-c or any other ordering ofa, b, and c).

No element, act, or instruction used should be construed as critical oressential unless explicitly described as such. Also, as used, thearticles “a” and “an” are intended to include one or more items, and maybe used interchangeably with “one or more.” Furthermore, as used, theterms “set” and “group” are intended to include one or more items (e.g.,related items, unrelated items, a combination of related and unrelateditems, and/or the like), and may be used interchangeably with “one ormore.” Where only one item is intended, the phrase “only one” or similarlanguage is used. Also, as used, the terms “has,” “have,” “having,”and/or the like are intended to be open-ended terms. Further, the phrase“based on” is intended to mean “based, at least in part, on” unlessexplicitly stated otherwise.

What is claimed is:
 1. A method for training models performed by a userequipment (UE), comprising: receiving, at the UE associated with a groupof UEs of a plurality of groups of UEs, a global model from a zoneserver associated with a first zone model of a plurality of zone models,the global model associated with a global server, the group of UEsassociated with the first zone model, and each group of UEs associatedwith a respective zone model of the plurality of zone models;identifying, at the UE, a device for training the global model based onone or both of a current connectivity state of the UE or a currentresource use of the UE, the device being the UE based on the currentconnectivity state of the UE satisfying a connectivity condition and thecurrent resource use satisfying a resource condition, the device being aproxy device based on one or both of the current connectivity state ofthe UE failing to satisfy the connectivity condition or the currentresource use failing to satisfy the resource condition; transmitting, tothe zone server, model updates based on identifying the device; andreceiving, from the zone server, the first zone model based on thetransmitted model updates.
 2. The method of claim 1 further comprising:training the model at the UE based on local training data collected atthe UE based on identifying the UE; and generating the model weightupdates based on the training.
 3. The method of claim 2, furthercomprising maintaining a local training state of the model acrosstraining iterations of the global model.
 4. The method of claim 2,further comprising: receiving a set of models at the UE, the globalmodel being one model of the set of models; and training the globalmodel from of the set of models based on a training priority of theglobal model being higher than a training priority associated with eachother model of the set of models.
 5. The method of claim 1, furthercomprising: transmitting the global model and training data collected atthe UE to the proxy device for training the global model at the proxydevice using the training data collected at the first UE; and receivingthe model updates from the proxy device based on the training performedat the proxy device.
 6. The method of claim 5, in which the proxy deviceand the UE are intra-network devices or inter-network devices.
 7. Themethod of claim 1, in which the global model is an untrained model. 8.The method of claim 1, in which the global model comprises weightupdates from one or more zone models of the plurality of zone models. 9.The method of claim 1, in which: the global model is a second zone modelgenerated at a second zone server associated with the second zone model;the global server is the second zone server; and one or more attributesof the first zone model and the second zone model satisfy a similaritycondition.
 10. The method of claim 1, in which the first UE and each UEin the first group of UEs share one or more common attributes.
 11. Themethod of claim 10, in which the one or more common attributes compriseone or more of a geographic location, a default language, or a userinterface theme.
 12. A system for training models, the system comprisinga user equipment (UE) and a zone server: the UE comprising: a processor;a memory coupled with the processor; and instructions stored in thememory and operable, when executed by the processor, to cause the UE: toreceive, at the UE associated with a group of UEs of a plurality ofgroups of UEs, a global model from the zone server associated with afirst zone model of a plurality of zone models, the global modelassociated with a global server, the group of UEs associated with thefirst zone model, and each group of UEs associated with a respectivezone model of the plurality of zone models; to identify, at the UE, adevice for training the global model based on one or both of a currentconnectivity state of the UE or a current resource use of the UE, thedevice being the UE based on the current connectivity state of the UEsatisfying a connectivity condition and the current resource usesatisfying a resource condition, the device being a proxy device basedon one or both of the current connectivity state of the UE failing tosatisfy the connectivity condition or the current resource use failingto satisfy the resource condition; to transmit, to the zone server,model updates based on identifying the device; and to receive, from thezone server, the first zone model based on the transmitted modelupdates.
 13. The system of claim 12, in which execution of theinstructions further cause the UE: to train the global model at the UEbased on local training data collected at the UE based on identifyingthe UE; and to generate the model updates based on the training.
 14. Thesystem of claim 13, in which execution of the instructions further causethe UE: to receive a set of models at the first UE, the global modelbeing one model of the set of models; and to train the global model fromof the set of models based on a training priority of the global modelbeing higher than a training priority associated with each other modelof the set of models.
 15. The system of claim 12, in which execution ofthe instructions further cause the UE: to transmit the global model andtraining data collected at the UE to the proxy device for training theglobal model at the proxy device using the training data collected atthe first UE; and to receive the model updates from the proxy devicebased on the training performed at the proxy device.